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ABSTRACT 

The Grammar, Spelling, and Punctuation (GSP) test is 
administered to students in the College of Communication at the 
University of Texas, Austin, as a means of determining eligibility to 
register for certain courses in journalism, broadcasting, and 
advertising. The test was administered in a pape r-and-penc i 1 version 
to 16 students and in a computerized (CAT) version of 23 students. 
Overall, results suggest that the CAT version is an excellent 
alternative to the paper~and-penci 1 test. The pass rate for both 
versions was 73%, and classification decisions of one version 
relative to the other had an overall agreement rate of 73%. Attitudes 
toward the clarity of directions, ease of use, and perception of 
skill assessment were overwhelmingly positive. An appendix lists 
equivalent scores on the two test versions. Six tables present study 
results. (SLD) 
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Results of the Administration of the Computerized Grammar, Spelling, 
and Punctuation Test to College of Communication Students 
on July 7, 1992 

Nancy J. Smith, Steven J. Fitzpatrick, 
and Barbara G. Dodd 

The Grammar, Spelling, and Punctuation (GSP) test is administered to students in the 
College of Communication as a means of determining eligibility to register for certain courses 
in Journalism, Radio-TV-Film, and Advertising. The lQO-question examination is offered at 
the beginning of each semester and prior to pre-registration periods during the fall and spring 
semesters. Students are administered one of six forms of the multiple-choice test in typical 
paper-and-pencil format 

In the field of psychometrics an increasing amount of research is being conducted in the 
area of computerized adaptive testing (CAT). Computerized adaptive testing procedures 
involve the repeated estimation of an examinee's ability level based on his or her responses to 
items which have been presented and the adaptation of further item selection according to each 
item's degree of difficulty and the examinee's latest ability estimate. Advantages of 
computerized adaptive testing over paper-and-pencil versions include administration of only 
those items appropriate for an examinee's ability level, shorter test length (usually one-fourth to 
one-third shorter) with equal or greater precision of measurement, and immediate feedback 
provided to the examinee. The purpose of this study was to compare examinees 1 test 
performance on the paper-and-pencil and CAT versions of the GSP and gain insight into their 
attitudes towards the two formats of the test. 

The GSP was administered in both paper-and-pencil format and computerized adaptive 
testing format on July 7, 1992. Volunteers to take the CAT version were solicited from the 
group of students who took the paper-and-pencil version of the GSP during the July testing 
period. 

Method 

Subjects 

During die scheduled testing period, 39 students took the paper-and-pencil version of the 
GSP. Of those, 23 students (59%) volunteered to take theCAT version of the test, as indicated 
in Table 1. One examinee took both versions of the GSP, but due to a handicapping condition, 
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a nonstandard administration of the paper-and-pencil version was required. For this reason, 
data obtained from this student were deleted from the analyses. 

Table 1 

Numbers and Percentages of Students Who Took the Various Versions 
of the Grammar, Spelling, and Punctuation Tests 
Given on July 7, 1992 
(N = 39) 



Version 


N 


% 


Paper-and-Pencil only 


16 


41.0% 


Paper-and-Pencil plus Computerized 


23 


59.0% 


Total 


39 


100.0% 



Note . One student who took both versions of the Grammar, Spelling, and Punctuation Test 
was deleted from subsequent analyses due to a handicapping condition that required a 
nonstandard administration of the paper-and-pencil version. 



Materials 

The paper-and-pencil version of the GSP consists of 100 questions — 33 questions each in 
the Grammar and Punctuation sections and 34 questions in the Spelling section. As stated 
previously, there are six forms of the test. 

All three sections of the GSP are printed in one test booklet for the paper-and-pencil 
version, and the examinees may take the sections in any order or work between sections, if 
desired. Examinees are allowed up to 90 minutes to complete the test, though they may turn in 
their materials earlier if they finish before time is called. 

The item bank for the CAT version was developed from the pool of items contained in the 
six forms of the paper-and-pencil version. The items span a wide range of difficulty levels. 

Examinees may take the sections in any order on the CAT version; however, they cannot 
move between sections. Each section must be completed before moving on to the next one. 
Within each section, items are selected and presented in a tailored fashion based upon each 
examinee's performance on previously administered items. Specifically, the first item 
administered to each examinee is randomly selected from those items which have a difficulty 
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level in the middle range. If the examinee responds correctly to the first item, the next item 
presented will be more difficult than the first; however, if the examinee responds incorrectly, 
an easier item is presented next. This process of adapting the presentation of items to match the 
examinee's skill level based on responses to the previous items is continued until a sufficiently 
accurate estimate of the student's skill is obtained or a predetermined maximum number of 
items have been administered within a section. The result is that students are administered test 
items that are appropriate for their skill level; consequently, they are not administered items that 
are either too easy or too difficult for them. For this reason, students are not typically required 
to take as many test items with the computer adaptive version as they are with the paper-and- 
pencil version. For this study, the prespecified maximum number of items per section 
corresponded with the limits on the paper-and-pencil version: 33 items for the Grammar and 
Punctuation sections and 34 items for the Spelling section. 

The two versions of the GSP also differ in terms of the scale on which scores are 
reported. The scores on the paper-and-pencil version are reported as a number correct score, 
while scores for the CAT version are reported as scaled scores that range from 20 to 80. For 
the paper-and-pencil version, a total score cf 72 is considered to be passing. Based on the 
results of previous research, a composite score of 45 on the CAT version has been shown to be 
equivalent to a total score of 72 on the paper-and-pencil version of the GSP. The composite 
score on the CAT version is the average of the scaled scores from the Grammar, Spelling, and 
Punctuation sections. 

Procedure 

On July 7, 1992, the paper-and-pencil version of the GSP was administered to 39 
students who were required to take the test prior to registering for certain College of 
Communication courses. Invitations to take the computerized adaptive version of the GSP 
were given to all examinees. Incentives used to solicit volunteers included no charge for the 
CAT administration and the use of the higher score between the CAT and the paper-and-pencil 
versions. The CAT version was administered at the Measurement and Evaluation Center to 23 
students. All examinees were presented with instructions via computer and allowed time to 
become familiar witn the keyboard and the appropriate method of responding to questions. As 
stated previously, examinees were able to select the order in which they took the three sections, 
but they were required to complete each section before moving on to the next one. It should be 
noted that no time limit was imposed on any section of the CAT version of the GSP. Results 
were presented to the student through the computer immediately after completion of the three 
sections. 
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After the students completed the computer adaptive version of the GSP test, they were 
asked to provide feedback about the test by responding to a set of five questions. These 
questions were also presented via the computer. 

Results of the Test Administration 

The average time spent on the CAT version, from presentation of the first screen of 
instructions to completion of the test, was 42.35 minutes. The average time spent on the 
examination itself was 37.18 minutes. For the Grammar section of the test, the average time 
examinees took was 16.39 minutes, while for Spelling the average was 5.96 minutes, and for 
Punctuaticn the average time was 14.83 minutes. These means are presented in Table 2, along 
with corresponding standard deviations, and minimum and maximum times spent on each 
section of the test and the sections combined. 

Table 2 

Descriptive Statistics for the Test Scores, Number of Items Administered, 
and Testing Times from the Computerized Version of the 
Grammar, Spelling, and Punctuation Test 
Given on July 7, 1992 



(N = 22) 


Scale 


Mean 


SD 


Minimum 


Maximum 


Composite 


49.73 


5.96 


39.00 


59.00 


NIA 


60.86 


10.16 


53.00 


100.00 


TT 


37.18 


9.60 


24.23 


55.25 


Grammar 


48.91 


6.90 


34.00 


62.00 


NIA 


19.16 


3.71 


17.00 


33.00 


TT 


16.39 


5.57 


8.85 


33.72 


Spelling 


49.04 


7.19 


34.00 


64.00 


NIA 


20.41 


3.70 


18.00 


34.00 


TT 


5.96 


2.06 


2.72 


11.30 


Punctuation 


51.14 


9.08 


35.00 


66.00 


NIA 


20.50 


4.48 


17.00 


33.00 


TT 


14.83 


5.08 


9.45 


32.13 



Note . NIA is the number of items administered and TT is actual testing time. 

The average number of items administered and the average scores per section and for the 
composite obtained from the CAT administration are also presented in Table 2, as are other 
descriptive statistics. The average number of items administered for the composite GSP was 
60*86, while the averages for the three sections were 19.96 for Grammar, 20*41 for Spelling, 
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and 20.50 for Punctuation, On the paper-and-pencil version, all examinees are required to take 
100 items. 

The average composite score was 49.73, with a score of 45 considered to be passing. 
The average Grammar score was 48.91, while the average score for Spelling was 49.04 and 
for Punctuation it was 51.14. 

In comparison to the paper-and-pencil version, the overall classification accuracy of 
pass/fail decisions based on the computerized version was 73%, as indicated in Table 3. 
Seventy-three percent of the examinees had a composite score of 45 or higher on the CAT 
version and 73% has a score of 72 or higher on the paper-and-pencil version; however, the 
students who passed one version were not necessarily the same students who passed the other 
version. Of the students who passed the paper-and-pencil version, 59% also passed the 
computerized version, while 14% who failed one version also failed the other version. Of 
those who passed the paper-and-pencil version, 14% failed the CAT version, while another 
14% passed the CAT yet failed the paper-and-pencil version. 

Table 3 

Computerized Decision Relative to Paper-and-Pencil Decision for 
Students 1 Grammar, Spelling, and Punctuation Tests 
Given on July 7, 1992 
(N = 22) 

Computerized 



Paper- 
and- 
Pencil 





Pass 


Fail 


Total 


Pass 


13 


3 


16 




59% 


14% 


73% 


Fail 


3 


3 


6 




14% 


14% 


27% 


Total 


16 


6 


22 




73% 


27% 


100% 



Note . Overall classification accuracy is 73%. 

Table 4 presents descriptive statistics for those 16 examinees who took only the paper- 
and-pencil version. The average total score for those who passed (N = 10) was 78.70, while 
the average score for the 6 who did not pass the paper-and-pencil version was 59.17. 
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Table 4 

Descriptive Statistics for the Paper-and-Pencil Total Test Scores Earned by Students 
Who Elected Not to Take the Computerized Version of the Grammar, Spelling, 
and Punctuation Test Given on July 7, 1992 
(N = 16) 



Pass-Fail Status 


N 


Mean 


SD 


Minimum 


Maximum 


Pass 


10 


78.70 


5.08 


73 


90 


FaU 


6 


59.17 


3.89 


53 


64 



A conversion table (see Appendix A) developed in a previous study was used to obtain 
CAT score equivalents for those 16 examinees who did not take the computerized version. 
When comparing the classification decisions of the computerized version relative to the paper- 
and-pencil version for the entire sample (N = 38; one examinee was deleted due to nonstandard 
administration), the overall classification accuracy rate is 84% (as shown in Table 5), which is 
comparable to a previous study. 



Table 5 

Computerized Decision Relative to Paper-and-Pencil Decision for 
Students' Grammar, Spelling, and Punctuation Tests 
(N = 38) 



Computerized 



Paper- 
and- 
Pencil 





Pass 


Fail 


Total 


Pass 


23 


3 


26 




61% 


8% 


68% 


Fail 


3 


9 


12 




8% 


24% 


32% 


Total 


26 


12 


38 




68% 


32% 


100% 



Note . The 16 students who elected not to take the computerized version of the GSP Test were 
included in this table by using the conversion table developed in an earlier study. The overall 
classification accuracy is 84%. 
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Student Feedback 

Students were presented five questions concerning their attitudes about the computerized 
version of the test after ihey completed the CAT version of the GSP. These questions and the 
distribution of responses are presented in Table 6. The vast majority of examinees felt that the 
directions were clear (95.4%) and that the method used to respond to the items was easy to use 
=(95.5%). Ninety-five and one-half percent (95.5%) of the examinees indicated that they liked 
getting their scores immediately after finishing the test (typically, examinees must wait until the 
next day to receive their paper-and-pencil scores), and 77% felt that the computerized version 
was able to assess their skills accurately. When asked which version of the GSP would be 
preferable, 50% indicated the computerized version, 36.4% indicated the paper-and-pencil 
version, while 13.6% had no preference. 

Table 6 

Response Numbers and Percentages for the Five Evaluative Items Pertaining to the 
Computerized Version of the Grammar, Spelling, and Punctuation Test 

Given on July 7, 1992 
(N = 22) 



The directions for the computerized version were clear. 



Strongly Agree 


14 


(63.6%) 


Agree 


7 


(31.8%) 


Neutral 


1 


(4.5%) 


Disagree 


0 


(0.0%) 


Strongly Disagree 


0 


(0.0%) 


The arrow keys used to enter my answers were easy to use. 






Strongly Agree 


17 


(77.3%) 


Agree 


4 


(18.2%) 


Neutral 


1 


(4.5%) 


Disagree 


0 


(0.0%) 


Strongly Disagree 


0 


(0.0%) 


I liked getting my scores immediately after the test was finished. 






Strongly Agree 


20 


(90.9%) 


Agree 


1 


(4.5%) 


Neutral 


0 


(0.0%) 


Disagree 


1 


(4.5%) 


Strongly Disagree 


0 


(0.0%) 
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Table 6 (continued) 

I think that the computerized version assessed my skills accurately. 



Strongly Agree 


9 


(40.9%) 


Agree 


8 


(36.4%) 


Neutral 


2 


(9.1%) 


Disagree 


3 


(13.6%) 


Strongly Disagree 


0 


(0.0%) 



Which version of the GSP would you prefer to take? 

Computerized 1 1 (50.0%) 

Paper-and-Pencil 8 (36.4%) 

No preference 3 (13.6%) 



Conclusion 

Overall, the comparison of performance on both versions of the GSP and attitudes about 
the computerized adaptive version of the GSP relative to the paper-and-pencil version was very 
positive. The pass rate for both versions of the test was 73%, and the classification decisions 
of one version relative to the other had an overall rate of agreement of 73%. It should be noted 
that the overall classification accuracy rate of 84% (when the paper-and-pencil scores of the 
people who did not take the CAT were converted to the CAT scale and included) is identical to 
the accuracy rate achieved from a previous simulation study with the GSP which compared the 
CAT version to the full scale (362 item) version. The attitudes towards the clarity of 
directions, ease of use, and perception of skill assessment were overwhelmingly positive. In 
summary, it appears that the computerized adaptive testing version of the GSP provides an 
excellent alternative to the paper-and-pencil version. 

The results of this study and another study conducted in 1992 were presented to the 
College of Communication. Based on these results, the Measurement and Evaluation Center 
was given approval to replace the paper-and-pencil administration of the GSP with the CAT 
version. The decision was made to limit the maximum number of items administered via CAT 
to 25 per section. More items did not provide additional information about a student's pass/fail 
status. Also, it was decided that examinees would be allowed to use scratch paper during the 
Spelling section due to the number of requests made by students during the July 7, 1992, test 
session. The administration of the CAT version of the GSP was implemented during 
September 1992. 



APPENDIX A 

Equivalent Scores on the Computerized and Paper-and-Pencil 
Versions of the GSP Test 



i 

APPENDIX A 

Equivalent Scores on the Computerized and Paper-and-Pencil 
Versions of the GSP Test 



Computerized 


Paper-and-Pencil 


20 


32 


21 


33 


22 


35 


23 


A ^ 

3o 


24 


38 


25 


40 


26 


41 


27 


42 


28 


44 


29 


46 


30 


48 


31 


49 


32 


50 


33 i 53 


34 | 55 


35 


57 


36 


58 


37 


59 


38 


*1 


39 


63 


40 


64 


41 


65 


42 


67 


43 


OS 


44 


70 


45 


72 


46 


73 


1 47 


74 


'A 


76 


49 


77 


50 


78 


51 


80 


52 


80 


53 


82 


54 


83 


55 


84 


56 


84 


57 


85 


58 


86 
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Appendix A (continued) 



Computerized. 


Paper-and-Pencil 


59 


87 


60 


88 


61 


89 


62 


90 


6$ 


90 


64 


91 


65 


92 


66 


92 


67 


93 


68 


93 


69 


94 


70 


94 


71 


94 


72 


95 


73 


95 


74 


% 


75 


96 


76 1 


96 


77 


97 


80 


97 
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